UNCLASSIFIED 


Defense  Technical  Information  Center 
Compilation  Part  Notice 

ADPO 10894 

TITLE:  Distributed  Collaborative  Virtual  Reality 
Framework  for  System  Prototyping  and  Training 

DISTRIBUTION:  Approved  for  public  release,  distribution  unlimited 


This  paper  is  part  of  the  following  report: 

TITLE:  New  Information  Processing  Techniques  for 
Military  Systems  [les  Nouvelles  techniques  de 
traitement  de  T information  pour  les  systemes 
militaires] 

To  order  the  complete  compilation  report,  use:  ADA391919 

The  component  part  is  provided  here  to  allow  users  access  to  individually  authored  sections 
of  proceedings,  annals,  symposia,  ect.  However,  the  component  should  be  considered  within 
the  context  of  the  overall  compilation  report  and  not  as  a stand-alone  technical  report. 


The  following  component  part  numbers  comprise  the  compilation  report: 
ADPO  10865  thru  ADP010894 


UNCLASSIFIED 


Distributed  Collaborative  Virtual  Reality  Framework  for  System 

Prototyping  and  Training 


29-1 


Suleyman  Guleyupoglm 
Henry  Ng 

U.S.  Naval  Research  Laboratory 
4555  Overlook  Avenue  SW 
Washington,  DC  20375 
United  States  of  America 

{suleyman,  ng}  @ait.nrl.  navy,  mil 


Introduction 

The  significant  impact  Information  Technology  has 
made  in  the  lives  of  most  people  in  the  last  decade  or 
two  is  undeniable.  Coupled  with  other  enabling 
technologies,  many  tasks  get  accomplished  more 
efficiently  and  reliably  nowadays.  However,  there 
are  instances  where  the  adoption  of  these 
technologies  takes  longer  for  variety  of  reasons. 
System  prototyping  and  training  of  war  fighters  has 
been  an  area  where  enabling  technologies  can  be 
better  utilized. 

One  of  the  keys  to  the  success  of  military  operations 
with  the  least  amount  of  casualties  is  a well-trained 
group  of  people  fighting  the  war.  It  is  also  essential 
that  they  are  well  accustomed  to  the  environment  in 
which  they  operate  and  comfortable  with  it.  To 
achieve  this,  live  training  would  be  ideal  in  almost  all 
cases  except  where  it  would  pose  a significant  safety 
risk.  This,  of  course,  is  not  possible  in  reality  as  the 
resources  available  for  training  is  limited  in  the  real 
world. 

In  this  paper,  we  address  the  use  of  enabling 
technologies  in  the  military  and  describe  a framework 
that  takes  advantage  of  several  state-of-the-art 
technologies  to  perform  combat  system  design, 
evaluation  and  training.  The  framework  uses  virtual 
reality  and  distributed  computing  technologies  to 
provide  users  (trainees,  trainers,  or  combat  system 
designers)  an  immersive  environment  to  interact  with 
the  combat  system  and  the  other  users.  The 
framework  allows  participants  to  collaborate  on  the 
same  mission  or  activity  in  the  virtual  environment 
without  the  need  to  be  at  the  same  geographical 
location. 


There  are  numerous  benefits  to  systems  based  on  this 
framework.  It  is  possible  to  bring  together 
participants  into  the  same  virtual  environment  as 
opposed  to  doing  this  in  the  physical  world  that  may 
be  costly  and  sometimes  difficult  logistically.  The 
cost  of  training  is  significantly  cheaper  since  the 
initial  cost  and  maintenance  of  computer  equipment 
is  usually  less  than  the  cost  of  real  combat  systems. 
The  framework  allows  new  weapon  systems,  for 
example  a combat  information  center,  to  be  built 
virtually  so  that  users  can  conduct  a walk-through 
and  make  suggestions  on  the  design  of  the  system.  In 
other  words,  the  end  users  can  actively  participate  in 
the  early  design  phase  of  a new  combat  system. 

Virtual  Engineering 

We  use  the  term  Virtual  Engineering  to  describe  the 
engineering  process  primarily  performed  in  a Virtual 
Reality  (VR)  environment.  Virtual  Engineering  can 
be  utilized  at  various  phases  of  the  development  life 
cycle  including  requirements,  design,  development, 
testing  and  training.  Virtual  Engineering  allows  C41 
concepts  and  combat  systems  to  be  evaluated  before 
they  are  physically  built,  resulting  in  significant 
savings  for  the  US  Department  of  Defense. 

There  are  a number  of  enabling  technologies  that  we 
use  to  make  Virtual  Engineering  effective  (See 
Figure  1).  At  the  top  of  the  list  is  Virtual  Reality. 
Virtual  Reality  technologies  allow  users  to  immerse 
themselves  into  the  environment  they  are  working  in. 
What  should  be  noted  is  that  3D  graphics  is  not 
always  Virtual  Reality.  What  makes  the  experience 
almost  real  or  virtually  real  is  usually  the  human 
computer  interface  accompanied  with  realistic 
computer  graphics. 


* Also  affiliated  with  ITT  Industries,  Alexandria,  Virginia,  USA. 
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Modeling  and  simulation  has  found  wide  acceptance 
in  many  engineering  fields.  Using  modeling  and 
simulation,  engineers  can  verify  the  validity  of  their 
design  and  optimize  them.  As  such,  modeling  and 
simulation  is  an  indispensable  tool  for  engineers. 

Distributed  Computing  or  Simulations  is  another 
technology  that  is  important  for  the  Virtual 
Engineering  of  combat  systems.  Different  simulators 
that  the  engineers  need  may  have  been  designed  to 
run  on  different  hardware  and  they  may  be  running  at 
different  physical  locations.  Therefore,  the  ability  to 
link  all  these  simulators  seamlessly  is  significant  for 
the  Virtual  Engineering  infrastructure. 

Parallel  Processing  is  also  an  important  aspect  of 
modem  engineering  process.  For  high  precision 
modeling  or  for  relatively  complicated  problems, 
existing  computing  platforms  can  be  used  in  parallel 
to  solve  problems  faster  than  the  alternative  of  using 
any  one  of  the  computers.  Some  problems  are  better 
suited  for  parallel  processing  than  others  are. 
However,  significant  run-time  reductions  can  be 
obtained  by  parallel  processing  in  general. 

Finally,  collaborative  environments  enable  engineers 
to  work  on  projects  synchronously  or  asynchronously 
from  geographically  distant  locations.  Traditionally, 
the  travel  costs  discourage  teams  of  engineers  who 
are  at  different  cities  or  countries  to  get  together  and 
work  together  as  frequently  as  they  would,  had  they 
been  at  the  same  location.  Collaborative 
environments  offer  a solution  to  the  problem  by 
allowing  engineers  to  perform  video  teleconferencing 
and  share  documents  or  applications  at  the  same  time. 

Virtual  Engineering  offers  a number  of  advantages 
over  traditional  engineering  practices.  With  Virtual 
Engineering,  combat  systems  can  be  well  understood 
before  resources  are  committed  for  building  the 
system.  The  users  or  the  warfighters  can  be  involved 
in  all  phases  of  the  development  process  early  on. 
Their  feedback  can  save  time  and  money  by  reducing 
development  life  cycle  and  total  development  time. 
Virtual  Engineering  serves  as  an  integration  tool  for 
design,  development,  integration  and  testing  of 
systems.  In  addition,  it  can  be  used  to  train 
warfighters  on  the  system  before  the  systems  are 
actually  built. 

Virtual  Engineering  is  well  suited  for  Simulation 
Based  Design  and  Acquisition  (SBD/SBA) 
applications.  Typical  SBA  life  cycle  can  be  divided 
into  four  phases:  requirement  gathering,  design, 
development  and  testing  (See  Figure  2). 


In  the  requirement  phase,  important  issue  decisions 
are  made  in  the  theater  context.  For  example,  what 
kind  of  force  mix  is  considered,  number  of  weapon 
systems  to  be  used  in  the  theater,  their  search  and 
detection  coverage,  reaction  time,  fire  power, 
capacity  and  weapon  load. 

In  the  design  phase  of  the  SBA,  decisions  are  made  in 
the  platform  context.  For  example,  system 
configuration  and  connectivity  decisions  are  made 
based  on  system  requirements.  These  decisions  are 
then  used  to  determine  the  types  of  algorithms,  logic 
and  display  components  to  use  at  element  context. 
This  requires  simulations  that  start  with  simple  logic 
for  the  requirement  formulation  phase  in  the  early 
acquisition  process.  As  the  system  engineering 
process  progresses  into  design  and  development 
phases,  the  fidelity  and  number  of  entities  in  the 
simulation  will  be  greatly  increased  to  meet  the  needs 
of  system  design,  development  and  testing.  However, 
much  of  the  model's  fidelity  often  has  to  be  sacrificed 
in  order  to  reduce  the  computational  turnaround  time 
and  meet  the  project  schedule.  In  other  situations,  the 
fidelity  of  the  model  needs  to  be  increased  to  gain 
more  data  and  insights  for  the  problem  under  study, 
increasing  the  model's  running  time.  When  running 
on  a single  processor  the  simulation  faces  the  "Von 
Neumann  Bottleneck".  By  taking  advantage  of  recent 
advances  in  low  cost,  high  speed  computer  chips, 
scalable  parallel  processors  and  new  software 
languages,  it  is  possible  to  have  fast  turnaround  times 
while  still  allowing  an  increase  in  model  fidelity  and 
simulation  scenario  complexity.  A composable 
simulation  model  which  employs  an  optimistic 
parallel  processing  architecture  in  which  the  number 
of  processing  nodes  can  be  scaled  up  or  down  as 
needed  was  developed  to  meet  this  need.  This  enables 
the  analysts  to  run  more  experiments  or  parametric 
analyses  to  find  the  optimal  system  configuration 
design  within  a run  time  constraint. 

The  next  phase  of  the  SBA  process  is  the  testing  of 
the  system.  An  additional  benefit  of  Virtual 

Engineering  is  that  the  testing  system  can  also  be 
used  as  a training  tool  for  the  end-users  of  the  system 
and  let  the  real  users  of  the  system  provide  feedback 
early  in  the  life  cycle. 

It  is  important  to  evaluate  different  components  with 
different  characteristics  in  the  development  process. 
For  example,  to  achieve  a certain  detection  coverage 
characteristics  a ship  may  carry  different  types  and 
different  number  of  radar  systems.  The  simulation  of 
these  radar  systems  can  help  identify  which  system  is 
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a better  solution  to  satisfy  the  requirements.  Being 
able  run  these  simulations  in  parallel  may  accelerate 
system  development  time  significantly. 

To  practice  Virtual  Engineering,  an  object-oriented 
set  of  classes  has  been  developed  at  the  Naval 
Research  Laboratory  (NRL). 

Virtual  Reality  Framework 

An  object-oriented  set  of  classes  has  been  developed 
at  the  NRL  to  form  a framework  for  Virtual 
Engineering  and  virtual  reality  applications.  The 
objective  of  the  framework  is  to  allow  rapid 
development  of  these  applications  with  significant 
software  reuse.  We  used  this  framework  to 
demonstrate  several  concepts  that  we  present  in  the 
paper. 

Virtual  reality  is  a natural  interface  between  humans 
and  computers,  where  communication  can  take  the 
form  of  moving  3D  imagery,  sound,  and  even 
physical  forces  (from  motion  to  touch).  Virtual 
reality  applications  can  be  so  responsive  and  render 
imagery  in  such  high  speed  that  the  users  can  feel 
immersed  in  the  synthetic  environment.  Many  virtual 
reality  applications  share  common  tasks  such  as 
rendering  geometric  models,  interfacing  with  the 
users,  communicating  with  other  users  on  the 
network,  etc. 

The  Virtual  Reality  Framework  (VRF)  provides 
several  classes  that  help  carry  out  those  tasks, 
independent  of  the  type  of  application  developed.  For 
example,  it  implements  classes  that  provide  rendering 
functionality  using  SGI  Iris  Performer  [1],  Similarly, 
it  implements  classes  that  let  the  application 
programmer  to  use  High  Level  Architecture  (HLA) 
Run-Time  Infrastructure  (RTI)  for  communicating 
with  other  networked  applications  or  simulations 
with  minimal  understanding  of  the  HLA  RTI  [2-4], 

We  present  two  applications  that  were  built  using  the 
VRF. 

Virtual  Prototyping 

We  used  VRF  to  develop  a virtual  prototyping 
application.  This  tool  allows  system  designers  to 
design  and  configure  the  equipment,  such  as  combat 
consoles,  large  screen  displays,  to  formulate  the 
layout  of  room  for  the  ship  command  center  in  a 
collaborative  environment.  System  designers  can  run 
the  software  on  a set  of  networked  computers  to  work 
on  the  same  system.  This  process  is  illustrated  in 
Figure  3.  Each  person  sees  the  workspace  as  shown 


in  the  lower  left  of  the  figure.  They  may  have 
different  types  of  user  interfaces  to  interact  with  the 
software  that  provides  different  levels  of  immersion 
into  the  virtual  environment.  These  interfaces  range 
from  a simple  set  of  keyboard  and  mouse  to  a more 
advanced  interface,  such  as  a workbench  or 
stereoscopic  head  mounted  displays  with  hand  and 
body  movement  hacking  devices. 

This  can  serve  as  a digital  mockup  of  the  ship 
command  center.  In  addition,  the  components  that  are 
placed  in  the  virtual  ship  command  center  is  linked  to 
simulations,  so  it  is  possible  to  interact  with  the 
consoles  in  the  command  center  while  running  the 
simulations  that  the  command  center  system  depends 
on.  In  other  words,  it  is  possible  to  conduct  virtual 
testing  of  the  configuration  using  this  virtual 
environment.  The  same  system  can  also  be  used  to 
train  crewmembers.  The  costly  physical  mockup  used 
for  ship  combat  system  design  and  development  can 
be  eliminated  while  reducing  the  development  and 
testing  time.  More  importantly,  the  system  allows 
design  experts  from  different  disciplines  as  well  as 
the  end  users  (warfighters)  to  work  as  a team  and  to 
interact  simultaneously  during  the  design.  Conflict 
and  design  errors  can  be  detected,  and  resolved  at  the 
early  stages  of  the  design  process.  This  is  especially 
important  because,  as  studies  at  the  US  Defense 
System  Management  College  have  shown,  the 
majority  of  a system’s  life  cycle  cost  is  determined 
very  early  in  the  design  phase  of  the  program. 

Having  designed  the  ship  command  center  in  the 
virtual  prototyping  program,  we  are  able  to  load  the 
configuration  file  into  another  application  for  testing 
and  training.  We  describe  that  application  in  detail 
next  in  a training  tool  context. 

Virtual  Combat  Information  Center  Training 
System 

Virtual  reality  technology  has  become  a cost- 
effective  training  alternative  with  the  advent  of  more 
powerful  and  cheaper  graphics  computers.  This 
technology  allows  the  user  to  be  immersed  into  a 
simulated  graphical  environment  where  he  or  she 
sees  the  virtual  environment  through  a computer 
monitor  or  a head  mounted  display  and  provides 
input  through  various  devices  such  as  a keyboard, 
mouse  or  more  sophisticated  alternatives.  We  used 
VRF  to  develop  a distributed  virtual  CIC  for  the 
surface  ship  training.  CIC  crewmembers  can  then 
perform  testing,  team  learning  and  training  in  a 
virtual  ship  environment.  The  virtual  environment 
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offers  a true  interactive  3D  view  of  the  interior  of  the 
CIC  as  shown  in  Figure  4. 

The  visual  simulation  portion  of  the  Virtual  CIC  is 
intended  to  provide  the  “look  and  feel”  of  the  actual 
CIC  through  the  use  of  extensive  3D  models  and 
sound  clips.  Each  student  appears  in  the  environment 
as  a 3D  human  representation,  an  avatar.  A picture  of 
each  student’s  face  is  scanned  and  texture  mapped 
onto  his  avatar  so  that  each  student  can  recognize  the 
others  inside  the  Virtual  CIC.  Networking  capability 
of  the  Virtual  CIC  allows  students  at  different 
geographical  locations  to  train  together  in  a unified 
virtual  environment  over  a wide  area  network.  This 
networking  capability  not  only  allows  the  crews  in 
the  same  ship  CIC  to  be  distributed  at  different 
locations  for  training,  it  also  provides  the  capability 
to  simulate  the  whole  virtual  battle  group  operations 
as  shown  in  Figure  5 so  that  different  command  and 
control  structures  can  be  developed  and  evaluated. 

Various  information  visualization  aids  are 
incoiporated  into  the  virtual  environment  to  help 
crewmembers  in  understanding  and  learning  different 
tactical  deployment  of  the  combat  systems.  For 
example,  a holocube  allows  students  to  visualize  the 
entire  battlespace  in  3D  as  shown  in  Figure  6.  This 
allows  crewmembers  to  correlate  sensor  data  with  a 
visual  3D  display  of  a god’s-eye  view,  facilitating 
understanding  of  sensor  capabilities  and  operations. 
This  holographic-like  display  could  also  allow  for 
visualization  of  sensor  coverage,  emission 
restrictions,  operational  boundaries  and  other 
doctrinal  concepts  and  entities.  As  the  crewmember 
changes  his  watchstation  console  mode,  the  holocube 
view  could  change  to  reflect  the  performance  of  that 
mode.  This  visual  augmentation  can  assist  the 
understanding  and  teaching  tactical  deployment  of 
offensive  and  defensive  capabilities.  Virtual  consoles 
are  easily  reconfigured  or  rearranged  as  needed  for 
different  ship  configurations  for  maximum  efficiency. 
By  combining  the  virtual  environment  with  a ship 
simulation  model,  operational  procedures  under 
various  tactical  scenarios  can  be  explored  and  refined 
easily.  System  performance  can  be  stressed  in  a 
realistic  scenario  without  the  risking  accidents  or 
tying  up  the  real  operational  hardware.  Incorporated 
information  visualization  aids  allow  ship  crews  to 
leam  and  understand  the  operations  much  faster  than 
in  a regular  classroom  environment.  Virtual  reality 
based  training  also  allows  students  to  train  with 
advanced  systems  before  the  hardware  has  been  built. 


or  try  out  new  physical  layouts  that  do  not  currently 
exist,  as  well  as  experiment  with  and  develop  new 
operational  procedures.  The  system  described  can  be 
applied  to  land-based  or  mobile  command  centers  as 
well  as  the  ship  CIC. 

Summary 

In  this  paper,  we  described  the  approach  and  benefits 
of  virtual  engineering  in  a distributed  environment. 
We  also  illustrated  two  applications  that  have  been 
developed  based  on  a virtual  reality  framework  to 
perform  virtual  engineering,  system  evaluation  and 
training.  The  system  provides  an  integrated 
environment  for  ship  design,  prototyping,  and  testing. 
This  will  enhance  ship  design  and  operations  by 
exploring  different  configurations  using  virtual 
prototyping  techniques  coupled  with  the  physics 
based  models.  It  can  also  provide  means  to  assess 
human  computer  interfaces  (HCI)  while  designing 
various  ship  system  components.  More  importantly, 
this  same  virtual  environment  can  be  used  as  (1)  an 
assessment  tool  to  evaluate  a candidate  design’s 
performance;  (2)  as  a means  to  letting  the  ship’s  crew 
have  an  early  visualization  and  experience  of  the  new 
design  and  provide  feedback  to  the  engineers;  (3)  as  a 
training  tool  for  the  ship  crew  before  and  after  the 
actual  physical  system  is  constructed.  This  virtual 
environment  can  support  all  phases  of  the  ship  life 
cycle.  This  approach  will  permit  a much  more 
realistic  assessment  of  a candidate  design  and  system 
configuration  early  in  the  design  process.  Changes 
can  be  made  earlier,  and  therefore  easier  and  cheaper, 
than  in  engineering  production. 
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Figure  1:  Virtual  Engineering  Technologies 
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Figure  2:  Virtual  Ship  Combat  System  Component  Development  for  Acquisition  Process 
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Figure  3:  Virtual  Prototyping  in  a Team  Setting 


Figure  4:  Virtual  Ship  Combat  Information  Center 
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Figure  5:  Virtual  Battle  Group 
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Figure  6:  Holocube  Visualization  Aid 


